Application No.: 10/670,245 
Art Unit: 2621 



Amendment under 37 C.F.R. §1.116 
Attorney Docket No.: 03 1 1 98 



REMARKS 

Reconsideration of this application, as presently amended, is respectfully requested. 
Claims 1-22 are pending in this application. Claims 1-22 stand rejected. 

Initially, it is noted that original claim 22 depends from claim 1. However, original claim 
22 should properly depend from claim 21. Therefore, claim 22 has been amended to depend 
from claim 21. 



Request to Withdraw Finality of Office Action 

As will be discussed below, withdrawal of the finality of the present Office Action 
requested because independent claim 21 has not been amended during the prosecution, and has 
now been rejected over a new ground of rejection. 

More specifically, regarding finality of an Office Action, the Manual of Patent Examining 
Procedure §706.07(a), states: 

Under present practice, second or any subsequent actions on the merits 
shall be final, except where the Examiner introduces a new eround of 
rejection that is neither necessitated bv Applicant's amendm ent of the 
claims nor based on information submitted in an Information Disclosure 
Statement filed during the period set forth in 37 CFR 1 .97(c) with the fee 
set forth in 37 CFR1.17(p). 



Furthermore, a second or any subsequent action on the merits in any 
application or patent undergoing reexamination proceedings will not be 
made final if it includes a rejection, on newly cited art other than 
information submitted an Information Disclosure Statement filed under 
37 CFR 1.97(c) with the fee set forth in 37 CFR 1.1 7(p), of any claim 
not amended bv Applicant or patent owner in spite o f the fact that 
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other claims may have been amended to require newly cited art . 

[Emphasis added] 

Independent claim 21 has not been amended during the prosecution of this application. 
Claim 21 was rejected in the previous Office Action under §103 over Chakraborty et al. in 
view of Gonsalves. Claim 21 is now rejected under §103 over Chakraborty et al. in view of 
Toklu et al. and further in view of Gonsalves. 

Therefore, because the rejection of claim 21 is a rejection on newly cited art of a claim 
that was not previously amended, the rejection of claim 21 constitutes a new ground of rejection 
that was not necessitated by Applicants' amendment of the claims. 

Accordingly, under the requirements of §MPEP 706.07(a), the finality of the current 
Office Action is improper and should be withdrawn. Withdrawal of the finality of the Office 
Action is respectfully requested. 

Claim Rejections - 35 U.S.C. S103 

Claims 1-6, 9-14 and 16 are rejected under 35 U.S.C. §103(a) as being unpatentable over 
Chakraborty et al. (USP 7,110,454, previously cited) in view of Toklu et al. (USP 6,549,643). 
Claims 7-8 are rejected under 35 U.S.C. §103(a) as being unpatentable over Chakraborty et al. 
in view of Toklu et al. and further in view of Blanchard (USP 6,347,1 14, previously cited). 
Claims 15 and 17-20 are rejected under 35 U.S.C. §103(a) as being unpatentable over 
Chakraborty et al. in view of Toklu et al. and further in view of Park et al. (USP 6,597,738, 
previously cited). Claims 21-22 are rejected under 35 U.S.C. §103(a) as being unpatentable over 
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Chakraborty et al. in view of Toklu et al. and further in view of Gonsalves (USP 6,392,710, 
previously cited). 

Initially, it is noted that in the current rejection of independent claims 1, 4, 9, 13 and 14 
(and dependent claims 2-3, 5-6 and 16) under §103, the Office Action substantially repeats the 
rejections from the previous Office Action, except the Examiner now applies the newly cited 
Toklu et al. reference to teach the claimed "shot segmentation device to segment the video into 
respective shots." 

More specifically, the Office Action asserts 

Chakraborty does not explicitly teach a shot segmentation device to 
segment the video into respective shots. However, Toklu teaches a shot 
segmentation device to segment the video into respective shots (video 
segmentation module 12, column 5, lines 38-57, and Fig. 1, element 12). 
See Office Action, page 8, lines 6-9. 

Toklu et al. discloses a video segmentation module that partitions a video file (either 
stored or input as a real-time video stream) into a plurality of video segments. The video 
segmentation method used by Toklu et al. is the cut detection method that partitions video data 
into a set of shots comprising visually abrupt cuts or camera breaks. See col. 5, lines 37-60. 

The segmenting of video data in Toklu et al, is performed so that a key frame (i.e., a 
frame to represent the content of a given shot) can be selected for each shot based on key frame 
selection techniques (see col. 1, lines 42-45; and col. 5, lines 61-65). Key frame selection 
techniques of Toklu et al. include analyzing camera motion between consecutive frames in a 
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shot, computing pixel based absolute frame differences between frames in a shot, and computing 
a color histogram of each frame (see, e.g., col. 5, lines 65 - col. 6, line 7). 

The other references applied against the claims were discussed in previous responses, and 
a detailed discussion will not be reiterated here. The independent claims will be addressed 
separately below. 



Claim 1 has been amended to clarify the difference between a shot and a scene. In 
particular, the amendments to claim 1 clarify that the scene is a larger unit than a shot, and the 
classified scene is smaller in number than a total number of shots. 

First, it is submitted that the Toklu et al. reference is no more relevant than the 
Chakraborty et al. reference. More specifically, the Examiner relies on Toklu et al to teach 
segmenting a video into respective shots; however, Chakraborty et al. also teaches 
segmenting a video into units of shots (see col. 5, lines 1-2). Further, if the shot segmentation 
device of Toklu et al. is combined with the system of Chakraborty et ah, the result would be 
a system that segments video into shots and then detects a key frame for each segmented shot. 

However, it is respectfully submitted that neither Chakaborty et al. nor Toklu et al. 
disclose or suggest the processing performed on the respective [segmented] shots (i.e., processing 
performed after the shots are segmented) in order to classify a scene. 

The Examiner provides an extensive Response to Arguments (see pages 2-6, Items 1-15 
of the Office Action). However, it is respectfully submitted that the Response to Arguments 



Claim 1 
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does not address the main argument previously presented that Chakraborty et al. does not 
disclose or suggest any of the processing that is performed on the segmented shots (i.e., 
processing that is necessarily performed after the video is segmented into shots). 

That is, in accordance with what was previously argued, Chakraborty et al. does not 
disclose or suggest "a shot segmentation device [that segments] the video into respective shots" 
and, after the video has been segmented into shots, "a calculator [calculates] shot density DS of 
the video from said respective shots" and "a calculator [calculates] motion intensity of the 
respective shots"; and "a dynamic/static scene classifier [classifies] the respective shots into a 
dynamic scene with much motions or a static scene with little motions based on the shot density 
and the motion intensity [of the respective shots]" [Emphasis added]. 

For example, in the Response to Arguments, the Examiner asserts: 

In response to applicant's argument that the references fail to show 
certain features of applicant's invention, it is noted that the features upon 
which applicant relies (preprocessing the detection of shot boundaries to 
classify the video in into units of shots before obtaining shot density and a 
motion intensity) are not recited in the rejected claim(s). [Emphasis 
added.] See Item 3 of Office Action. 

The above assertions set forth in Item 3 of the Office Action have no basis and are 
without merit. First, the claims clearly recite the features (preprocessing the detection of shot 
boundaries to classify the video into units of shots before obtaining shot density and a motion 
intensity) upon which applicants rely. That is, the language of claim 1 clearly defines the 
invention such that the "calculator for calculating shot density," the "calculator for calculating 
motion intensity" and the "dynamic/static scene classifier" process (i.e., perform the recited 
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operations on) the respective shots that have been (previously) segmented (i.e., preprocessed) by 
the shot segmentation device. 

Moreover, the Examiner continues to assert that the various interframe difference 
metrics of Chakraborty et al. correspond to the various elements recited in claim 1. 

More specifically, in Item 5 of the Office Action, the Examiner asserts: 

Chakraborty discloses examining frame-to-frame intensity changes as the 
pixel level (column 1 line 53-54); pixel activity would indicative of motion. 
Chakraborty teaches an integrated process for segmenting video shots by 
successfully identifying scene changes, both abrupt and gradual. 
A combination of a plurality of difference metrics are processed in an 
integrated fashion to identify and verify both gradual and abrupt scene 
changes in the video (column 4 line 66 to column 5 line 5). Further, 
Chakraborty discloses a predefined shot duration (column 13 line 15 to 
35); which is equivalent to the shot density. Since Chakraborty discloses 
to examine the frames at a pixel level, and segmenting the video into both 
gradual and abrupt scenes, and the shot duration is used with detecting 
the abrupt scene (column 13 line 13-35), it is clear to the examiner 
that Chakraborty segments the video into abrupt or gradual scene 
based on motion, which reads upon the claimed limitation. 

However, the Examiner's response set forth in Item 5 of the Office Action ignores the 
fact that, as was discussed in the previous response, (1) Chakraborty et al. disclose that a 
"scene" and a "shot" are defined as the same thing (see, e.g., col. 5, lines 44, 52 and 62) and (2) 
detecting a "scene change" is the same as a detecting a "cut" point (see col. 13, lines 15-35). 

Thus, it is clear that determining "scene changes" in Chakraborty et al. is the same as 
identifying cut points so that shots can be determined (or segmented). Determining a "scene 
change" (i.e., cut point) in Chakraborty et al. is not the same as the claimed classifying of the 
respective shots into a dynamic scene or static scene. In particular, identifying a cut point (i.e., 
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scene change) in Chakraborty et al. is a precursor to segmenting a shot. In other words, the 
shot has not yet segmented at the point the cut point is determined. Therefore, identifying a cut 
point cannot classify a shot, as claimed. 

Further, contrary to the Examiner's assertions, it is clear that the use of the various 
metrics in Chakraborty et al. to identify "scene changes" can not be the same as the claimed 
"metrics" used to classify a scene. More particularly, the metrics in Chakraborty et al. are 
used to identify a cut point so that a shot can be subsequently segmented. In other words, the 
metrics in Chakraborty et al. are not used to process a shot that has been segmented from 
video. In contrast, the claimed "calculator for calculating motion intensity," "calculator for 
calculating shot density" and "dynamic/static scene classifier" are used to analyze (i.e., calculate 
shot density, calculate motion intensity, classify into a dynamic or static scene) the segmented 
shots. 

In view of the foregoing, reconsideration and withdrawal of the rejection of claim 1 are 
respectfully requested. 



With respect to claim 4, it is submitted that none of the cited references disclose or 
suggest the claimed "slow scene detector for classifying the target shot into a slow scene of the 
similar shot based on motion intensity of the target shot and the similar shot" 

The Examiner apparently considers the gradual scene change disclosed by Chakraborty 
et al. (see col. 7, lines 49-50) to correspond to the claimed "slow scene detector". See Office 



Claim 4 
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Action, page 9, last four lines). However, the "gradual scene change" disclosed by Chakraborty 
et al. is not a shot. The gradual scene change is a portion between detected scenes. Therefore, 
Chakraborty is not classifying a target shot. 

Further, Chakraborty et al. does not disclose or suggest that the "slow scene" is detected 
based on motion intensity of the target shot and the similar shot. 

Claim 9 

With respect to claim 9, in the previous response, it was agued that Chakraborty et al. 
does not teach performing operations on segmented shots, specifically, "a detector for detecting a 
histogram relating to motion directions of the respective [segmented] shots." Furthermore, it was 
argued that the claimed "histogram relating to motion directions of the respective shots" is 
different from the histogram difference metric of Chakraborty et al., which is related to pixel 
intensity distribution of an image. These arguments are hereby reiterated. 

In Item 3, last sentence, the Examiner concludes "Chakraborty discloses examining pixels 
at a frame-to-frame intensity changes at the pixel level, which would [be] indicative of motion 
(column 1, lines 53-54), Chakraborty discloses a predefined shot duration (column 13, lines 15- 
35), which is equivalent to shot density." 

It is noted that Chakraborty et al. teaches that interframe difference methods of 
detecting scene changes (cuts) are particularly sensitive to object motion and camera motion (see 
col. 2, lines 57-60). However, the statement that interframe difference methods are sensitive to 
motion, does not teach that these interframe difference methods detect motion direction. 
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Chakraborty et al. is silent with respect to detecting motion direction using the interframe 
difference metrics. 

Moreover, arguments similar to those presented above with respect to claim 1 are also 
appropriate. More particularly, the various metrics disclosed in Chakraborty et al. are used to 
identify a cut point so that a shot can subsequently be segmented. In other words, the metrics in 
Chakraborty et al. do not analyze, or perform operations on, a segmented shot. That is, the 
various metrics of Chakraborty et al. do not disclose or suggest "a detector for detecting a 
histogram relating to motion directions of the respective shots" because the shots are not 
segmented when the various metrics are applied to the frames of the video. 

Accordingly, reconsideration and withdrawal of the rejection of claim 9 are respectfully 
requested. 

Claim 13 

Regarding claim 13, it was previous argued that Chakraborty et al. do not disclose 
or suggest "a detector for detecting a shot density DS of the video" and "a commercial scene 
detector for detecting a commercial scene by comparing a shot density detected during a 
predetermined interval with a predetermined reference shot density." 

The Examiner had not previously responded to these arguments. The Examiner now 
responds to these arguments in Items 8 and 9 of the Office Action. Specifically, the Examiner 
asserts: 
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Regarding applicants['] argument that Chakraborty does not disclose the 
"detector for detecting a shot density DS of the video" and "a commercial 
scene detector for detecting a commercial scene by comparing a shot 
density detected during a predetermined interval with a predetermined 
reference shot density". 

The examiner respectfully disagrees. Chakraborty discloses video are 
playing an increasingly import role in education and commerce, 
Column 1 line 15-18. Further, when the approximate maximum duration 
is known, since the frames/sec is always known, the maximum frame 
duration for the scene change is readily ascertainable. If any of the 
windows have a duration that exceeds this threshold, it may be assumed 
that the window in question is not likely to be a gradual scene change. In 
such as case, further examination becomes necessary. The possibilities are 
that either the window represents just motion or a combination of scene 
change and motion. In the preferred embodiment, if any window has a 
duration that exceeds the predefined threshold, it is assumed that the 
window represents motion, and consequently all points in such window are 
turned "off (step 224). All the remaining windows are then identified as 
candidates for gradual scene change, column 14 lines 20-35. 
Chakraborty teaches a predefined shot duration (column 13 line 15 to 
35); which is equivalent to the shot density. Therefore, since 
Chakraborty discloses videos in education and commerce, and based on 
the predefined window threshold, the scene is either gradual or abrupt, it 
is clear to the examiner that Chakraborty is fully capable of detecting a 
commercial scene based on the shot density, which reads upon the claimed 
limitation. 



As best understood, it appears that the Examiner believes that "shot duration" is the same 
as "shot density". The Examiner cites col. 13, lines 15-35 of Chakraborty et aL, which relates 
to determining whether a distance between any two abrupt scene changes is less than a 
predetermined minimum shot duration (see Fig. 2B, step 215 and col. 13, lines 24-28). 

First, it is submitted that "shot duration" is not the same as "shot density." However, 
based on the assertion that "shot duration" is not the same as "shot density," the Examiner relies 
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on a portion of Chakraborty et al. that teaches verifying whether the duration of each individual 
scene change (cut) is too large (see col. 14, lines 10-35) to teach "comparing a shot density 
detected during a predetermined interval with a predetermined reference shot density." 

More specifically, col. 14, lines 10-35 of Chakraborty et al. teaches comparing the 
duration of a detected (candidate) scene change with a predetermined threshold duration (e.g., 
one or two seconds). If the duration of the detected scene change is greater than the 
predetermined threshold duration, the duration of the individual scene change is determined to be 
too large, and it is assumed that the (candidate) scene change is motion, and is not a scene change 
(see col. 14, lines 29-33). 

However, it is submitted that, contrary to the Examiner's assertions, "shot duration" 
(described in col 13) and "scene change duration" (described in col. 14) are not the same as 
"shot density." 

Accordingly, reconsideration and withdrawal of the rejection of claim 13 are respectfully 
requested. 

Claim 14 

In the previous responses, it was argued that Chakraborty et al. does not disclose or 
suggest detecting a commercial scene based on a number of shot boundaries detected. The 
Examiner had not previously responded to these arguments. The Examiner now responds to 
these arguments in Items 10 and 1 1 of the Office Action, as follows: 
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Regarding applicants[ 5 ] argument that the reference does not disclose or 
suggest detecting a commercial scene based on a number of shot 
boundaries detected. 

The examiner respectfully disagrees Chakraborty discloses video in 
education and commerce, column 1 line 16-18. As defined by Merriam- 
Webster Dictionary, a commercial has to do with commerce. Further, the 
intensity range of a given frame is divided into a number of predefined bins, 
with each bin corresponding to an intensity range. Next, the number of 
pixel in each bin counted to generate the corresponding distribution 
comprising the histogram, column 8, lines 51-55. 

Applicants agree with the Examiner's general assertion that a commercial has to do with 
commerce. Otherwise, the Examiner's position is not understood. Specifically, it is not 
understood how the intensity range of a given frame relates to "a commercial scene detector for 
detecting a commercial scene by comparing a number of shot boundaries detected during a 
predetermined interval with a predetermined reference number r The intensity range of a given 
frame is unrelated to shot boundaries and comparing a number of shot boundaries detected 
during a predetermined interval with a predetermined reference number. 

In any event, the Examiner has not pointed out where Chakraborty et al. explicitly, 
implicitly or inherently discloses "a commercial scene detector for detecting a commercial scene 
by comparing a number of shot boundaries detected during a predetermined interval with a 
predetermined reference number" 

Therefore, the rejection of claim 14 is improper and should be withdrawn. 
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Claim 21 

In the previous response, it was argued that neither Chakraborty et aL nor Gonsalves 

disclose or suggest "extracting and combining means for extracting and combining a plurality of 
highlight scenes" and "inserting means for inserting a video transition effect into a combined 
portion of the respective highlight scenes, wherein the inserting means makes a type of the video 
transition effect to be inserted different according to whether the highlight scenes to be combined 
are the dynamic scene or the static scene." 

The Examiner responds to these arguments in Items 12-15 of the Office Action as 

follows: 

Regarding applicants['] argument that the combination of Chakraborty 
and Gonsalves does not disclose "extracting and combining means for 
extracting and combining a plurality of highlight scenes". 

The examiner respectfully disagrees. Chakraborty teaches after the 
different scenes (shots) are determined, the scene change validation module 
20, as an additional validation process, extracts keyframes for each of the 
shots and compares neighboring keyframes. If the keyframes are not 
sufficiently different, the corresponding shots are merged, column 7 line 
52-56. Chakraborty discloses to merge the corresponding shots of the 
keyframes that are not sufficiently different, it is clear to the examiner that 
the corresponding shots merged from the keyframes that are not sufficiently 
different, reads upon the claimed limitation; as understood by the 
examiner a highlight scene can either be abrupt or gradual. 

Regarding applicants argument that the combination of Chakraborty and 
Gonsalves does not disclose an "inserting means for inserting a video 
transition effect into a combined portion of the respective highlight scenes, 
wherein, the inserting means makes a type of the video transition effect to 
be different according to whether the highlight scenes to be combined are 
the dynamic or the static scene." 
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The examiner respectfully disagrees. Chakraborty discloses a transition 
between two shots is made in a gradual manner using special editing 
machines to achieve a visually pleasing effect. These types of gradual 
changes are also called "optical cuts". There are several types of optical 
cuts, such as "fade in", "fade out", "dissolve", "wipe", "flips", 
"superimpose", "blow-ups", "move-ins", etc. column 1 line 55-61. 
Gonsalves teaches a non-linear video editing system, with the capacity to 
display and modify video frame on a field-by field basis, has the capacity 
to implement special effects on a field-by field basis, thus increasing the 
accuracy of the effect, column 3 line 10-13. Since Gonsalves discloses to 
implement special effects on a field by field basis, it is clear to the 
examiner, that Gonsalves would be fully capable of inserting different 
effects based on the type of scenes, which reads upon the claimed 
limitation. 



As best understood, the Examiner interprets the merging of two neighboring scenes 
(shots) if keyframes extracted from the shots are not sufficiently different (see Chakraborty et 
al. col. 7, lines 52-60) to correspond to the claimed "extracting and combining means for 
extracting and combining a plurality of highlight scenes." 

However, it is submitted that the rejection is improper because Gonsalves does not teach 
inserting a different type of video transition effect according to whether the highlight scenes to 
be combined are a dynamic scene or a static scene. As noted above, the Examiner asserts "Since 
Gonsalves discloses to implement special effects on a field by field basis, it is clear to the 
examiner, that Gonsalves would be fully capable of inserting different effects based on the type 
of scenes, which reads upon the claimed limitation." 

However, the standard under §103 is not that a reference "would be capable" of 
performing a certain claimed feature. The reference must disclose or suggest the claimed feature. 
The Examiner does not point out where Gonsalves teaches inserting a different type of video 
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transition effect between combined highlight scenes according to whether the highlight scenes to 
be combined are a dynamic scene or a static scene. 

Therefore, the rejection of claim 21 is improper and should be withdrawn. 

The remarks above address the independent claims. However, the dependent claims also 
patentably distinguish over the cited references by virtue of their dependency on the independent 
claims. 



In view of the foregoing, it is submitted that all pending claims are in condition for 
allowance. A prompt and favorable reconsideration of the rejection and an indication of 
allowability of all pending claims are earnestly solicited. 

If the Examiner believes that there are issues remaining to be resolved in this application, 
the Examiner is invited to contact the undersigned attorney at the telephone number indicated 
below to arrange for an interview to expedite and complete prosecution of this case. 



CONCLUSION 
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If this paper is not timely filed, Applicants respectfully petition for an appropriate 

extension of time. The fees for such an extension or any other fees that may be due with respect 

to this paper may be charged to Deposit Account No. 50-2866. 

Respectfully submitted, 
Westerman, Hattori, Daniels & Adrian, LLP 

William M. Schertler 

Attorney for Applicants 

Registration No. 35,348 
Telephone: (202) 822-1100 
Facsimile: (202) 822-1111 

WMS/dlt 
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